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ABSTRACT 



In tha spring of 1974, e questionnmri containing Alabami R2R's 26 orlttria for a 
sucee^ftil reading program was sent to a sample of iducators and eduGator-relatad groups 
in the state of Alabama, The purpose of the questionnaire was to dettrmlni tht degree of 
consensus among thise groups about the Importance of the 26 eriterii for suectss* The rt- 
spondtnts were asked to rate the importance of the criteria on a scale of 1 (extremely im- 
portant) to 5 (no importance). The results of this round of the survey were then sent out 
to the initial group of respondents with a request to again rate the criteria, knowing this 
time how the other respondents had rated them. The results of the survey are as foilows: 
The total group of Round 1 resporidents (N - 918) rated half of the criteria of extremely 
hjgh importance and half of high Importance and were quite variable in their judgment 
(The majority of the criteria had a standard deviation of 1,0 to U,). In Round 2, the re- 
spondents rated the criteria of the same degree of importance, but wart only slightly 
variable in their judgments (itandard deviation of .5 to .99). After the second round of 
the survey, a common population (those who responded to both rounds of the sun/ey) 
was hand sorted from the total pile of questionnaires to determine if there were any sig- 
nificant differences between their judgments and the judgments of the total population. 
There were none. It can be said, therefore, that the educator and the educator-rilatid 
groups in the state of Alabama rated the 26 criteria for a successful reading program of 
extremely high or high importance and that tfiey moved toward greater ■lomogeneity In 
their Judgment between Rounds 1 and 2, Additionally, they rated the same criteria of the 
same importance, thos# In tht top half of the ratings beipg In tfia Instruetlon and facilities/ 
materials categoric. 
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I/INTRODUCTION 



Alabama Becomes a Right to Read State 

The first significant step toward uniting the resources of Alabama, both financial 
and human, to achieve spicific reading goals and objectives was taken on November 6-7, 
1972, when the State Board of Education appointed Alabama's first Right to Rtad Ad- 
visory Commission. 

When Dr. Ruth Lovi Holloway, National Director of R2R, addressed the first 
meeting of the Advisory Commission on January 10. 1973, she indicated that Alabama 
would soon be officially disignated as a Right to Rtad state, qualifying it for a grant of 
$50,000 to establish an organizational structure at the state level. At this time, Dr. 
Holloway addressed the specific goals of R2R and presented the strategy for accomplish- 
ing them. 

The Alabama State Department of iducation responded by officially idopting 
the R2R Plan of Action and made commitments to the Implementation of its require- 
ment as a means of being dislpnated a Right to Read state. In compliance with the 
Office of Education, the State Department of Education submitted a formal proposal on 
January 17, 1973, which included a statement of activities propospd for tfie implementa- 
tion of the R2R Program. On June 6, 1973, Alabama received approval of its proposal 
and notification of Its 550,000 grant award. The purpose of this award was to coordinate 
existing reading functions by building comprehensive reading programs through the skills 
and competencies developed by staff training. The participating schools received no monies 
for personnel or materials. For this reason, Alabama, at the request of the Office of Edu. 
cation, applied for and received an additional grant of $82,000 to train at least one reading 
director in every school system In the state. 

Development of 4ia Criteria for Success • 

At the first training program in the fall subsequent to the twenty-day summer 
training session, it was decided to try to Identify the critical characteristics of a successful 
reading progrim. This procfdure, it was hoped, would provide state unity on the goals of 
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FIGURE 1 

SYSTIM FOR ACHIiVING STATI AND NATIONAL RIGHT TO READ GOALS 

ALASAMARIR 



R2R. Local educators in the 14 participatmg ichool districts, tht state advisorY commii* 
sion, the task force, and riiding specialists providtd input for an initial stt of 34 criteria 
(see Diagram 1), Theit criteria wire ivintuilly reflnid into a set of 26 tfiat were formally 
acctptid by the participating groups. (See the Delphi survey instrumant in AppendiK A 
for a list of the 26 criteria,) 

The Delphi Study 

This process of riflnlng and adopting the criteria primirily involved getting group 
consensus which reflected a rather high degree of approval. However, group consansui is 
not considered to repreiint strong perional support, but a weaker form of public accept- 
ance. It was felt that such public acceptance was not an adequate basis on v^ich to claim 
approval of the criteria from the entire state, particularly in view of the unique position 
Right to Read occupies as a leadership program with no financial leverage to encourage 
local program development. Strong private commitment to the criteria was necrssarv* 
To obtain this kind of commitment, statewide consensus was sought from educators by 
using a procedure that would yield reliable data utilizing a highly respected statistical pro^ 
cedure for determining consensus— the Delphi technique. The criteria were submitted to 
3,500 educator throughout the Alabama public ^hools, using tfie Delphi technique for 
obtaining group consensus. 

The purpose of this monograph Is to report the results of that Delphi survey. 
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II/THE DELPHI TECHNIQUE 



The Concept of tfie Delphi 

The Delphi technique cm be definid is a process to elicit and refine group 
jiidgments. 

The taihnfque was originally designid by RAND Corporation for the purpoie of 
using collective expertise to develop and refine policiei, plans, or ideas thit required In- 
formed Judgment. The end product is consensus on a given topic. 

The process has three basic steps. The first step is mainly concerned with eliciting 
the initial idtas, policies, plans, tte,, which will bt reflnfd In sttps two and three. This 
step is usually conducted by a group of tKptrts different from those responding to the 
materiaL The second step is the initial response of the participants. The third step Is the 
solicitation of a second response from the participants when they are given sel^ted feed- 
back on their initial group responses from step two. Step three^ used once, or repeated as 
many m four times, constitutes the refinement cycle that results In consensus. 

The steps in the procest may be summarized briefly: ^ ^ 

1, Elicit opinion . 

Elicit the opinions of experts, participants, or a selected audience about 
(a) specific, predetermined topic(s). 

2, Evaluate. 

Ask the participants to evaluate a total list related to the topic's) on the basis of 
some criterion, such as Importance or crlticalness, 

3, Re'evaluite. 

Return the list to each participant along with a suiNmafy of the responses for 
each Item given in step two and aik for re^evaluation or a reason for maintaining 
the initial choice if the participant's responie vamu from other participants. 

The responses given in steps two and three art usually obtained with a survey instrument 
of some type, often a questionnaire. 

The prMess Just described has three common features: A first feature is anonymous 
response, that is, no respondent knows the exact response of any other respondent to any 
Item on the questionnaire {or other survey instrument), A second feature Is interaction. 
This Is achieved through systematjc feedback to the respondents of the group respond 
In each successive round. This systematic feedback is related to the third HmrB.smtistlcgl 
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group respome^ The itatistics Includi such data as tht modi, rangi, and location of a 
partfcipant's response in relation to the risponsis of the entirt group. In somi cases, 
means and itandard deviatloni are included. 

These three features ovircome three major obstacles to aGhieving consensus: 

1. The effects of dominant Individuals , 

Group opinion is highly influenced by dominant Individuals who usually talk 
tfit most There Is very little correlation between the pressure of a group 
member's speech and his knowledge, 

2. Irrelevant input 

Much of the avowed problem-oriented discussion in group situations Is irrelevant 
or biasid because it is usually more concerned with individual and group inter- 
ests tfian with problem solving 

3. Group pressure toward conformi^ . 

Fice^tCNface interchange and confrontation create pr^sure toward conformity 
that can distort individual Judgmentr~"^ 

« 

Value of tfie Delphi Technlqut 

Tht Delphi teehnique has some special proptrtiti that tend to make its use in social 
systems desirable. It is a rapid and relatively efficient way to get significant data from key 
actors in a social system without having to use large amounts of time to detirmlne key 
or critical incidents in a system's development. It overcomfs the logistics problems of 
conflicting individual schedules which make group meetings imprictical, expensive, and 
unrealistic. It requires much less individual effort to respond to i well-dtfined survey 
instrument than to participate in a conference or write out a paper, particularly since 
attempting to achieve consensus in a fice-to-face situation may prevent the participant 
from giving his true view. Tht use of systematic procedures that art inhtrent in the tech- 
nique tends to decreise the possibility of spurious outcomes. Anonymity and group 
response on items of common Interest and concern provide for a sharing of responsibility 
while releasing respondent inhibitions. 

a . ■ * 

Uses of The Delp«ii Technique 

The Delphi technique was originally used In the 1960s as a method for organizing 
and iharing expert opinions about forecasts for the future. The technique sought to help 
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ditarmfne chronology of ^ientific and technology tvents and to Judge when the avints 
might occur baied on the sptculation of several experts," Since then, tht Delphi has been 
used in mdustry for forecasting technologlea I development and in other organizations for 
examining dicisions on policies in iducation, public transportation/ and public health. 

The Delphi technique has had wide application in industry, business, education, 
and many other related fields. Many reeent efforts have concentrated on the field of edu» 
cation in areas such as forecasting, goal identification and selection, and determining 
consensus. In the field of education, participant groups have included admini^ratoi^, 
alumni, community, faculty, students, trustees, parents, and advlsofy councils, to name 
only a few. * 

In most Delphi studies, the data have been organized and collected in a specific 
manner. Based on the purpose of the study, a survey instrument Is developed to determine 
expert opinion on specific items. These items might be concerned with such things as 
statements on goals or objectives or on fuftjre evente. In most cases, the survey Instrument 
contains an importance scale on which participant are expected to Indicate the importance 
of a goal, evf?rit, or other item{s). One common scale used in Delphi ^dles is a five-point 
scale from 1 (most important) to 6 (least important), (The order can be reveled, with 1 
standing for least important and 5 for most important.) Most Delphi studies require the 
participants to respond to the same items on repeated rounds. After each of the Initial 
and subsequent rounds, participants are given feedback from the entire group. The feed- 
back includes such data as the mode, range, and location of a participant's score on a given 
Item compared to the location of scores of the entire group. In some cases, additional 
feedback data also Include means and standard deviations. 

One of the expected results of Delphi studies is that mean scores on individual 
Items, as well as total scores, will tend to shift as rounds with the Delphi instrument(s) 
are conducted. There is generally variation in the degree and dlr^tlon of tfiift, depending 
on the nature of tiie study. (Forecasting may have a greater shift tfian goal consensuij In 
a study of Institutional goals using 252 pefions in higher education, 27 of whom were 
faculty, mean scores shifted between the first and tiilrd rounds on a suivey instrument 
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While the shift In mean scores Is often Inslgnlficint, the dtcreise In dlspe relon of 
scores in many Delphi studies Is significant. For the same group of 252 edueators men- 
tloned above in a study of institutional goals, the standard deviation decreased ftfr' 18 
items on an eighteen-item scale. ^ 
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This change is typical of moit Delphi studies. The standard deviation generally decreai 
between the first and subsequent rounds of interrogation. 
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Most Dtlphi studies indicate a variation in scores across individual ittmi. It might 
be ixpectid that any study which uses opinion and speculation to forecast future events 
or comment on df sired goals or objectives would exhibit a widt range of scort s. In subse- 
quent rounds, this range has a tendency to decrease, with the second round producing the 
greatest change. Recent studies on the number of repetitions or rounds to get a higher 
degree of consensus have tended to show, however, that most of the change occurs after 
the first modal distribution to reported to all participants and that additional rounds fail 
to produce any significant changes. As subsequent rounds are conducted, the change in 
the range decreases. In sum, fte responses to items on Delphi studies tend to vary con- 
siderably for the initial round and to a Itsser degree for subsequent rounds. 

Summary 

• 

The Delphi technique, then. Is a method of determining consensus. Using some 
type of survey instrument (often a questionnaire), participants evaluate items on the 
Instrument (often with a number), indicating the degree of importance of a given Itern 
based on their expert Judgment. These responses are analyied statistically to determine 
the degree of consensus. The statistical summary is then given to fach participant who 
again rates the survey Items. Knowing the degree of consensus, the Individual has the 
opportunity to modify or retain his initial response. By this means, consensus can be ap- 
proached without the Inhibiting, conformlty-inducing, time-consuming, and, sometimes, 
time-wasting procedure of face-to-face discussion. In the rounds following the initial 
pass, the greatest changes occur. Sometimes the mean decreases^ usually the standard 
deviation does. This technique originated In busmen and industry and has had applica- 
tions in other fields, including education. 
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m/THE DELPHI STUDY 



Background and Purpose of tiie Study 

Alabama Right to Read followed a national mandate to further improve the 
quality of its reading program statewide. In carrving out this mandate, Alabama desired 
to obtain statewide consensus on the goals of the program. A small group of those directly 
Involved in R2R-the advisory commission, the LEA R2R directors, selected LiA educa- 
tors, and the R2R task force-had developed a set of 26 criteria which they thought, 
based on their expert judgment oould be fundamental goals of Alabama's R2R effort.. 
But they wished to achieve consensus statewide for purposes of building commitment to 
and a base for evaluation of the state effort. This need for statewide consensus was critical 
because R2R monies were supplied for purposes of improving ejcisting reading programs. 
No monies were allocated to LEAs for staff; an additional grant from the U.S. Office of 
Education did, however, provide monies to train one LiA director in 14 participating 
school districts. In order to provide an effective bridge between the director and the dis- 
trict personnel, it was necessary to achieve communication and agreement on the purposes 
of the program. The Delphi offered a means for achieving such consensus. 

To achieve this consensus, a sample of educatore tfiroughout the state were sur- 
veyed. The groundwork for the development of this statewide survey had already been 
laid in the development of the criteria. All that was required was the development of a 
survey instrument that could be used as the basis of the Delphi technique. 



Research Questions 



The basic research question was: What degree of consensus exists In Alabama re- 
garding the 26 criteria for success among the various major educator and education-re- 
lated groups in the state? Specifically, What degree of consensus existed Initially among 
these groups? What change was brought about by the use of the Delphi technique itself? 
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Methodology 

In order to carry out the Delphi study, two steps weri neca^iry- (1) construction 
of a sample ind (2) the development of a survey instrument 

» 

The Sample 

Initlaliy, fourteen categories of educators or educatlon-relitid groups were identi- 
fied. These groups were randomly sampled, ranging from a minimum of 10 percent of 
the larger groups, such as teachers, to a maKimum of 100 percent of the smaller groups, 
such as local superintendents* The sample totaled 3,680 respondents. Table 1 prtsents 
the details of the sample. 



TABLE 1 

ALABAIVIA RIGHT TO READ DELPHI SAMPLi 
Round 1 





Total 
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Pergtnt 


RespDndent Orpup 


Number 
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Ritumed 


L Supoflntendent/Ai:st. Super^ 


126 


126 
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4i 
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2/ itf mentary Principal 


540 


B4 
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€1 


3, Middle/Jf, High Pfinelpal 
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3S 


4. High Schawl Prlnclpa! 
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I. LiA iupsrvlsef/Diffctor 
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Bi 
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SO 


6. Cufrint Li A R2R DIltrfaB 
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04 


7* Elementary Teacher 
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ili 
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41 


8. Middlf/jr. High Teacher 


4J00 


470 


10 


122 


26 


9. High School Teacher 


11.758 


1,768 


10 


168 


d 


10^ Hlghcf id^ Reading Initf. 


30 


30 


100 


16 


S3 


11p State R2R Advisory Comm. 


31 


31 


100 


5 


16 


12. State R2R Task Force 


14 


14 


100 


3 


31 


13. SDi Consultant 


BO 




BO 


9 


36 


14. Other 






sol: t 






Tatal/Average 


27,297 


3«580 


13.1 


910® 


26 



^Onty 902 of these vvtre usable. 



A total of 3,580 forms wert Sint out on Ftbruaiy 7, 1974, with i raqutit for rtturn by 
March 5, 1974, Of the 918 {25,6%) returrted, 909 {25,3%) wtm usable. The sample in- 
cluded all livils of educational manigement, taachei^, and those direotly Involved in the 
R2R effort (the districts, advisory commi^lon, task force), consultants, and reading In- 
structoi^ in inititutions of postsecondary education. These 619 responses con^itute 
total population r^onst of Round 1, i.e,, all tfiose who re^nded. 
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Only the 919 reipondents of Round 1 wtre stnt a lUiviy form in Round 2, A 
total of 599 raiponsis ware returned. Thesfconstituti tht total population responsi for. 
Round 2. 

A common population (thoit who reipondid to both roundi) was idintifiid afttr 
the Round 2 results were sent in. This population totaled B25, not 699. This was due to 
the failure of some respondents to fill in their name on the form or to the fact that a 
person different from the perton the form wa^ sent to filled It out These responses Gould 
be countid for the total population but not for the common population* 

The Survey Form 

The survey form used in Rounds 1 and 2 consisted of a listing of the 26 criteria 
for success developed and refined by the R2R directors, LEA educators, the R2R task 
force, and the R2R advisory commission in the fall of 1973. A copy of the instrument 
on which is marked tfie summary results obtained in Round 1, Is attached as Appendix A. 
This marked form was sent to the respondents for Round 2, 

The instrument asks the respondents to indicate the degree of Importance of each 
criterion on a five-point scale from extremely high importance (5) to no importance (I). 
A category of "don't know'' (0) was also provided as a possible response. The fomiat of 
tfie Items was like the following example: 



20p Proviilons yrt made for ttachmg every itudent 
at his owyn Instruetlonil level and fearning rite. 



intremtfy High Msd, Low No Don't 
jitfi Impoftanet Imp, Inip. Irnp* Imp. Know 



Respondents were asked to circle the number which b^t expressed their judgment 

In discussing the results of Rounds 1 and 2 of the survey, the following terms and 
definitions are used: 

4-^5 Extremely high Importance 

3-3*99 High importance 

2-2,99 IVIedlum importance 

1^1-99 Low Importance 

Less than 1 No importance 
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On the Item above, tha mean Round 1 reiponst was 4J2. In discussing the responses, this 
neport uses tha convention "according to the general (or totaD group of respondents, it is 
of extre.nely high importance that this criterion be part of the Alabama R2R program." 
In indicating the measure of central tendency, the results refer to the mean response, 
unless ^ecifled otfierwist. 

The measure of dispersion (the spread of scores about the mean) of responses used 
in this report is tite standard deviation (SD). In dlMussing differences of opinion or dis- 
persfon, tfie following conventions are used: 

90 MMning 

Abovil.B " Greatly varlflbla 

1.0to1.B Quite vtrlabia 

,iOto.ifl Slightly vifiable 

Below .60 Good ■greemtnt 

Again using item #20 as an example, the SD of responses on degree of importance for all 
respondents in Round 1 was .85. Using the conventions In these instances, it can be said 
that "our re^ondents were slightly variable in their views on tfia importtnce of criterion 
#20." '\ ' 
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IV/RESULTS; ROUNDS 1 and 2 

A copy of the Delphi survey instrument is included as Appendix A. Summiry 
statistics covering all items are Included as AppendiK B. Individual statistics for each of the 
14 respondent groups are included as Appendix C. The results of Rounds 1 and 2 of the 
survey ire discussed In this section. First the total population responses are dlaussed, 
then the common population; then the two populations are compared. 

Total Population 

•■- 

Round I 

Oyervlaw 

\ 

m 

The mean (average) responses for Round 1 are half in the very high importance 
category and half In the high importance citegory. This meins that half of the criteria are 
considered of very high importance to the Alabama R2R educators and half are considered 
of high importance. The range of means is from 4.82 to 3.18. The majority of the standard 
deviations (20 of the 26) are in the quite variable category (1.0 to 1.BSD). Of the remain- 
ing six items, five are in the slightly variable category (.50 to .99SD), and one is in the 
greatly variable category (above 1.5). The range of standard deviations is from 1.60 to .70. 

Criteria Judged of Extremely High importance 

The criteria In this group art 1, 3, 8, 10, 12, 15, 18, 20, 21, 22, 23, 25, and 26. 
By ranl< they are as shown in Table 2. Three ties in rank occur: Individmlixed Instruction 
and Positive Environment are tied for first place with a mean of 4.62; Scope and Sequence 
of Learnings and Suppkmemry Reading Materials are tied for fourth place with a mean of 
4.35; and Training of Content Area Teachers and Individual Student Record-Keeping 
System are tied for seventh place with a mtan of 4.21. The range of means is from 4.62 
to 4.02. 
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TABLE 2 

ORITERIA JUDGED OF EXTREMELY HIGH IMPORTANCE 
ROUND 1 TOTAL POFULATiON 



Rank 


Critiflen 
Hq, 


Mtan 


Qrlttrion PMriptor 


1.5 


20 


4J2 


lfidiviriujiti?iiH Intifitrtliin 


1.B 


31 


4.62 


Fositivfi Envlfonmint 


3.0 


22 


4.37 


Tfsch^ Usi of Vfifious Riiding 


4.5 






Methods snd Tfchniquts 


1 


4.35 


Scope and Sequincs of Learning! 


4.S 

i.o 


23 


4.35 


Supplimintiry Rtading Milerlafs 


10 


4.26 


Variation of Student-Taagher Rati^ as 


7.6 






Neeasiary 


12 


4.21 


Training of Content Area Tea^hti^ 


7.5 


3 


4.21 


Indivldyal Studf nt Ra^rd- Keeping System 


9 


25 ' 


4.15 


Su^ortlve Medii 


10 


8 


#^4.11 


Madia Center 


11 


18 


4.0S 


Sharing of 1 nstruational Methods 


12 


15 


4.07 


LiA Director of All Reding Activities 


13 


26 


4.02 


Cental Location for Rtading Matarlats 



Crltarii Judged of High ImpQrtanoi 

The criterii in this group irt 2, 4, 5, 6, 7, 8, 11, 13, 14, 16, 17, 19, 24 By rank 
thty are as rfiown in Table 3. There is only one tie: Report to Parents and CtioTdimiion 
with Preschool Program are tied for siKtepnth place* 



TABLe 3 

CRITERCA JUDGED OF HIGH iMPORTANCE 
ROUND 1 TOTAL POPULATiON 





Orittrion 






Rank 


No, 


Mean 


Critirion Dmrlptor 


14 


2 


3.65 


Coordtnatipn and Articulation of All Speciai Reading Pf^ 
grams with the 6asl€ Currlaufum 


15 


14 


3.83 


Media Centir Staffed by Professional and Supportive 
Pirsonnil 


16.5 


16 


3.82 


Report to thef a^^nts 


16.5 


4 


3.82 


Coordination wltn Pretshool Prop^am 


18 


11 


3,81 


Continuous Staff Development 


19 


9 


3.79 


Complete LiA Testing System 


20 


24 


3.73 


Materials Rteognlie Variations in Raoa, Cultura, Sax 


21 


7 


3.63 


Ineintlves for Staff Development 


22 


19 


3.61 


Jr.^^ High Teasher Knowledge of Developinental Reading 


22 


6 


3.56 


Continuous Reading Program 


24 


17 


3.42 


Report to the Community 


25 


13 


3.40 


Twined Votynteer Helpen 


26 


5 


3eia 


Adult B^lo Iducation Reading Componant 
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Standard Devio^ugn in Round I 

When the standard deviations for Round 1 are considerid. It is possibli to define 
further the answef to the question ''How much consensui exists about the 26 criteria for 
success fn the total populition In Round V" 

As Indicated earlier, the respondents judged half of the criteria to be of extremely 
high importance and half to be of high Importance; but they are quite variable (1.0 to 
1.5SD) in their views on all but six criteria (see Table 4), For those six, they are slightly 
varied In their views about five (JO to .99SD) and greatly varied for one (SD above 1J). 
The criteria about which there IS slight ^riation are 20, 21, 22, 23, and 25. Of these, four 
(20, 21, 22, and 23) are also the top four in rank* As a group, they comprise all but one 
of the total criteria in the instruction categorv. 



TABLe4 

STANDARD DEVIATION 
ROUND 1 TOTAL POPULATION 





N0» lif Items 


Ittffl Nm, 


^1^5 Oretlly varlibit 


1 


It 


1.^1,S Quite variiblt 


20 


1*^18,24 

a0,2t,22,23,2§ 


M Sligfiliy verfibfv 


i 


^ i§ Oood agntmsnt 


0 




Total 


2e 





Discussion 

The iTJults of Round 1 show thit the total population who responded to the 
survey judge half of the crittrla to be extremely Innportant and half to be highly fmportint 
and that they are quite variable In their Judgment Tht Griteria which received the strongtst 
support are the criteria In the instruction and facilities and matirials categories. To ixpliini 

The 26 criteria were grouped by category when they were developed. The break- 
down and the number of criteria In each category are as follom: 



CrltiFien 








1. Ofiinlzitlen and Administritidn 




t-9 


A- Pfogfim . . .... 


9 


IMS 


S. Staff — . . 


i 


1&-18 




3 


19-22 


IL Instruction 


4 




III. FadlHlM and Mjtarlals 


_4 




Total ' 


2S 



In Round 1, the distribution by category of very highly supported and highly sup^ 
porttd oriterla are: 





^ Viry Highly 


Highly 






Sypportid 




Total 


QrganUation & Admlnlitration 








Progrsm 


3 


6 


9 


Staff 


3 


3 


a 


Qommunliv Rilatient 


1 


2 


3 


Irstructien 


3 


1 


4 


Faeilltlaf & Mitfrlaii 


-3 


-1 


4 


Column Total 


13 


13 


28 



Round II 

Sample 

Only the 919 reipondenti to Round 1 wart mcludad in the second mailing {see 
Table 1 for a breakdown by category). Of these, i99 {65%) responded, a high return rate. 
This rate is higher than the 26% return of Round 1. _ . 

k 

Survey Form 

The survey form shown in Appendix A, with the mean responses from Round 1 
entered on It, was sent to the respondents on April 1^ 1974^ with a request to return by 
April 26, 1974, 

Overview 

The rasponsis of the total group in Round 2 rfiow little changa in judgment about 
the dagraa of importance of criteria. Where changes do occur they are positive in all cases 
but one where the changa is so small as to be negligible. The standard deviation decreased 
for all criteria. Both of these changes indicate a greater consensui from Round 1 to 
Round 2, 

Criteria Judged of Extremely High Importance 

The criteria in this group are the same as in Round 1, with the addition of one, 
#14, which moved up out of the high importanaa category. By rank they are as shown 
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in Table 5, which also comparii the ranki with those of Round 1 and shows the amount 
of Ghange in the mean from Round 1 to Round 2* 

The major change in rank here Is the risolution of the ties from Round 1 for items 
20 and 21, 23 and 1, and 3 and 12* The means of all criteria all became more positive^ 
but by such a small percentage (+,14) as to be insignificant 



TABLE 5 

CRITiRIA JUDGED OF EXTREMiLY HIGH IMPORTANCE 
ROUND 1 vs. ROUND 2 TOTAL POPULATION 



Rank 
Round 1 


Rank 
Rpund 2 


Item No» 


Mean 


Ohange 


DtSGrlptor 


1.S 


1 


20 


4J4 


+.11 


Indivlduaflzed Instruction 


1.5 


2 


21 


4.73 


+J1 


pQsitivi Envlronmtnt 


3 


3 


22 


4.62 


+.26 


Teacher Usi of Various Readlni Methyl and 












Tichniquei 


4S 


4 


23 


4J8 


+.23 


Sypplemintary Rtadlng Matiriali 


4.S 


s 


1 


4,48 


+.11 


Ssopi and iiqutnci of Learnlnp 


6 


6 


10 


4.42 


+.16 


Variation of itudint-Ttaahir Ratios as Nec^ 












tary 


7.5 


7 


3 


4.34 


+.13 


fndlvldyal Student Reaord^Kfiplng System 


9 


8 


25 


4.33 


+.19 


Supportive Media 


7.5 


9 


12 


4,30 


+.08 


Training of Content Aria Tiaahirs 


10 


10 


8 


4.2i 


+*18 


Medli Center 


11 


11 


18 


4.17 


+.09 


Sharing of Instrustional Methods 


12 


12 


15 


4.1 S 


+.09 


LEA Pirtctor of All Riadlng Aetlvittes 


13 


13 


26 


4.14 


+.11 


Cintral Location for Rfidlng Materials 


14 


14 


14 


4.00 


+.17 


Media Center itaffed by Frofesiional and Sup' 



portlvi Pirsonnel 

Avfrage Change +*14 



Criteria Judged of High Importance 

The oritiria In this group are the same as In Round 1 with the ixcaption of #14, 
thecrltarlon that moved up Into the txtrimely high importance citegory. The ranklngi 
show more ihlftlng than tht Items in the ixtrtmely high Importanoe Citsgory. Tha averiga 
Incraasa in mtan (+.08) is again such a small psrcantaga as to be unimportant (Sat Table 6), 

Standard Deviation in Round 2 

The ilight increase in mean value for most of the criteria, whili not significant in 
itself, does indicate a movament toward greater consensus which, when coniidarad with 
the changes in standard deviation in Round 2, is significant. Standard deviation 
for all items decreased in Round 2, with an average deore^i of ,34. A eomparison of 
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TABup e 

CRITERIA JUDGED 0^ HIGH IMPORTANCE 
ROUND 1 vs. ROUND 2 TOTAL POPULATION 



Rank 
. Roond 1 


Rank 
Reund 2 


Ittm No. 


Mian 


Ohangt 


14 


1§ 


2 


3.pl 


+.05 


19 


16 


9 


3*90 


+.11 


16,5 


17 


16 


3.87 


+.05 


13 


18 


11 


3J7 


+.06 


16 J 


19 


4 


3.82 


+.00 


22 


20 


19 


3.81 


+.20 


21 


21 


7 


3.79 


+.11 


20 


22 


24 


3.78 


+.05 


23 


23 


e 


3.67 


+.11 


2S 


24 


13 


3.45 


+jg6 


24 


2i 


17 


3.39 


+.03 


26 


26 


s 


3.34 


+.15 






Averse Change 


+.08 



Dtserfptef 



Coerdlngtlon €i Articulation All Spt€ial Reading Pro« 

grams with the Biiig Currieyluin 
Complete LiA Testing System 
Report to the Parents 
Continuous Staff D^elopment 
Coordination with Prts^haol Progrimi 
Jn^Sr* High Teieher Knowledge of Dmtopmtnnl Ridding 
Insentlvts for Staff Developmint 
Materials RiEogniie Variations In Riee, Culture, Sex 
Continuous Rtadlng Program 
Traintd Volunttir Hilperi 
Report to ths Community 
Adult 8^iq idueitlon Riading Component 



TABLE 7 

STANDARD DiVIATION: ROUND 1 VS, ROUND 2 
TOTAL POnULATION 



SO Qategory 



^ I^S Greatly vartabie 

1*0^1.5 Quiie varidtite 

.99 Slightly variable 

^ 3 Good agr eemtnt 



Total 



RminH 
1 



I 

19 

a 

0 

26 



Round 

1 



0 
3 
23 
_0 

26 



Item No, 



5. 13. 19 
•II othefi 



tht number of items in each of the four stindird diviation citagories is shown in 
Table 7 above. 

In Round 2, the 1 Item in the greitly variible oitegory moved into the quite. viri- 
ibie citegorv and 17 items from the quite variable □ategory moved into the slightly virh 
able category. The range of standard deviation In Round 2 is from 1.08 to ,BB; in Round 1, 
from 1,60 to ,70, i decrease of one full category amount at the upper end point (1,60 — 
1.08 - ,52), The standard deviation in Round 2 Indicates primarily slight difference of 
opinion for ail items. 

Discussion 

The overall positive change in means (though in Itself insignificant J and stendard 
deviation, when taken together, indicate an increase In the degree of consensus about the 
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26 criteria for succiss imong tducator groups in Alabama from Round 1 to Round 2. The 
lick of change in the means is not untypical of Delphi studies. The change in standard 
deviation li more common. 

Whit thi change in standard deviition means essentially is that the divergence of 
the scores from the mean (or the spread of scores about the mean) has decreased^ This 
means more respondent agreemint about a given item, in this case, a criterion. 

This griater consensus is the goal of the Delphi technique* It means, in the case of 
Alabama's R2R effort, that the total group of educators feel that most of the 26 criteria 
for success are judged of high or very high importance md that there is only slight variation 
ibout this Judgment Such consensus i| necessary to succ^sful program implementation and 
operation, 

COMMON POPULATION 

Round 1 ^ 

Sample 

The common population consists of those 525 respondents v^ho filled out the 
survey form in both rounds of the survey. This list of B2B vi^as established by hand sorting 
through the total set of returned questionnaires. This procedure was followed in order to 
determine if any significant differences existed between those who responded to both 
rounds of the survey and those who responded only to Round 1; that is, if any bias 
existed in the common population. 

Overview 

The mean responses for Round 1 are half in the very high importance category 
and half in the high importance category. The range of means is from 4,70 to 3*23, The 
standard deviations indicate a fair amount of difference of opinion about the mean: 
1 is in ^e greatly variable category, 15 are in the quite variable category, and 10 are in the 
slightly virlable cattgor/. The range of itandard deviations Is from 1,57 to ,ifi. 



27 



Criteria Judg^ of Extremely High importanci 

The criteria in tfiis group are 1, 3, 8, 10, 12, 15, 18, 20, 21, 22, 23, 25, and 26, 
Thast comprise half of the 26 criteria. By rank they are as shown in Table 8* One tie 
oecurs-for first place-bttw^en numbers 21 and 20, Positive Envtronmmt and IndU 
vidualtzed Instmction. The range of means is from 4*70 to 4,07, ' 



TABLE S 

CRITERIA JUDGED OF EXTREMELY HIGH IMPORTANCE 
ROUND 1 COMMON POPULATION 



Rank 


Ne. 


Mi an 


13 


21 


4.70 


1.1 


20 


4.70 


3 


1 


4.43 


4 


22 


4.41 


5 


23 


4.37 


8 


10 


4.32 




12 


4.26 


9 


3 


4.22 


9 


8 


4.18 


10 


25 


4.17 


11 


18 


4,12 


12 


26 


4.08 


13 


IS 


4,07 



F^itrvt Envtrsnmint 

Individuallztd Iristruction 

Seopa and St qutnce Ltirninp 

Teaehtr Uss of VarlQut Raiding Methods and TidiniquM 

Supptimentary Raiding Matariaft 

Variation of Student^Tiachtr Ratioi ^ N8€9Mry 

Training of Content Arti Tiashtrs 

Indrviauil Student Reoord-Kiiping Syttem 

Media Centir 

Supportive Media ^ . 

Shgring of Instrwstlonal Methods 
Central Lpaation for Reading Materfali 
LEA Direetor of All Reading A^tivitiis 
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Criteria Judged of High importance 

The criteria in this group are 2, 4, 5, 6, 7, 9, 1 1, 13, 14, 16, 17, 19, and 24. Thoy 
comprise the remaining half of the 26 criteria* One tit occurs-between numbers 9 and 16, 
Complete LEA Testing System and Report to the Parents. The range of means is from 3 J9 
to 3.23. (See Table 9 on page 21.) 

Discussion 

If we consider the original grouping of the criteria Into categoriis of organization 
and administration, instruction, and facilities and materials, as shown in the Discussion 
section for the total population, the degree of support by category for these criteria Is 
exactly the same as for the total population because the same criteria are In the "ex- 
tremely high" and "high" Importance categories for each group. This would seem to 
indicate general agreemtnt betwein the total population and the common population 
about which criteria were of nhe highest importance. 

20 % 
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TABLE i 

CRITERIA JUDCED OF HIGH IMPORTANCf 
ROUND 1 COMMON POPULATION 





Qiteridn 
No, 




14 


2 


3,99 


«E 


4 


3.91 


16 


14 


3.89 


17.5 


9 


3.S5 


1#,i 


16 


3.8B 


IS 


11 


3.84 


20 


24 


3.77 


21 


7 


3.7i 


22 


19 


3^68 


23 


€ 


X55 


24 


17 


3.47 


2i 


13 


3p42 


2S 


§ 


3.23 



Critarian D^rlpter 



Coordmat!@n and Artfeutitldn of AW Sped^ Rttf log Pro- 

grgmi }N\th the BmiQ Curriculum 
CoordlnitiQn with PrMhoai Program 
Media Centf r Staf ftd by Prof clonal and Supportiva 

Pinonnel 
Oomplita LIA Tilting Systim 
Ripon to the Pirenti 
Cqntinuous Stiff Dtvilopmtnt 
MjttrlaJi Reeo^lza Vw-iitloni in Rm, Ouifun, Six 
Inetntivii for Stiff Deviloprnant 

Jr.^r. High Tiiohtr Knowltdga of Dsval^mantai Rmllng 

Continuous Reading Progrim 

Rtpoft to the Community 

Tralntd Volynteaf Halpirs 

Adult BmiQ Education Rtedlng Compontnt 



TABLE 10 

STANDARD DIVIATION 
ROUND 1 COMMON POPULATION 



SPCitegefy No. of Itam lutm 



> 1.S Oriitiy varlabla 1 19 ' 

LO-13 Quita variable IS 2,4, f, 7, 8, 9, 10, 11, ia 14, 1i, 16. 

24. 26 

.99 Slightly viHabIa 10 1, 3, 10. 12, 18, 20. 21. 22, 23, 26 

< M Good agreenriint 0 

Total 28 



Standard Deviation in Round 1 

The standard deviation for the common population, 3S it did for the total popula- 
tion, makes it possible to determint more clearly the degree of consensus which exists 
about the 26 criteria for success In the common population In Round 1. 

As indicated earlier, the common population respondents Judged half of the 
criteria to be of extremely high importance and half of high Importance, They art greatly 
variable in their agreement about 1 of these, quite variable In their agreement about IB, . 
and slightly variable about 10. The range of standard deviations is from 1.57 to J8. 

Discussion 

The result of Round 1 of the Delphi survey for the common population indicate 
tfiat the respondent consider half of the criteria of ejctremtly high imponance and half 
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of high Importanca and that they art only moderitaly variable in tfielr judgment. They 
are quite like thi total population in thtlr degree of consensus, exctpt ^at tfiey show le^ 
variibili^ in their fudgments because they have only IB criteria In the quite variable SD 
category, compared to the total population's 19; likewise, they have 10 criteria In tfie 
slightly variable SD categorv, compared to the total population's 8. 

Round 2 
Overview 

The rtsponses of the common population in Round 2 show little change in judg- 
ment about the degree of importance of the criterlat Where changes do occur tfiey are 
primarily positive, although not significantly so. The standard deviation decreased for ell 
criteria. 

Criteria Judged of Extremely High Importgnee 

The criteria in this group are the samR a^ in Round 1, with the addition of one, 
#16, Report to the PaFents. Bv rank they are as #iown in Table 11, which also compares 
the ranks with those of Round 1 and shows the amount of change in the mean in the Round 
2 responses* All means increased positively, but not significantly. Average increase is +,12. 
A number of criteria moved up in rank; noteworthy are those whose Increase in mean 
^ore is almost twice the average* These are criteria numbers 22, 23, and 26* As these 
moved up, others moved down. 

Criteria Judged of High Importance 

The criteria in this group are the same in Round 1, with the exception of #15, 
which moved out of the very high importance category. All criteria shifted in rank except 
#23 which remained 23rd in rank* A number of ties emerged in this rounds criteria 
numbers 16 and 9 tie for 16th place; criterias 4, 7, 19, and 24 are tied behind the 18th 
ranking criteria. All but three criteria increased in mean, but by insignificant amounts. 
The average increase is only +,04* 
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TABLf 11 

CRITERIA JUDGED OF EXTRIMELY HIGH IMPORTANCE 
ROUND 1 vs. ROUND 2 COMMON POPULATION 



Rink 


Rink 








Round 1 


Round 2 


Ittm No. 


Mein 


Changs 


1-i 


1,S 


21 


4.77 


+.07 




1,5 


20 


4.77 


+,07 


4 


3 


22 


4 63 


+.22 


S 


4 


23 


4.60 


+.23 


3 


5 


1 


4.48 


+.0i 


S 


6 


10 


4.45 


+.13 


10 


7 


25 


4.37 


+,20 


8 


8 


3 


4,33 


+.11 


7 


9 


12 


4.32 


+.06 


9 


10 


8 


4.28 


+•10 


11 


11 


18 


4.19 


*,07 


12 


12 


26 


4.17 


+.09 


13 


13 


15 


4.16 


+,09 


17,1 


14 


16 


400 


+.1i 



Dmilptsr 



Positive Envir5nm€nt 
Individualliid tnstruetlpn 

T€iahtr Ust sf Vsrlsuf fleading Mtth^s and T^niquM 

Supplemgntary Rtading hUtwnM\% 

Seopi ind Siquenat ef Ltarnlnp 

Varlitjon of Student-Teichir Rstioi m NM^ry 

iupponlve Mtdia 

Individual Sfydint Rteord-Kfiplng Svitem 
Trimmg of Contt nt Am Teachtti 
Medti Center 

S.hiring of Initmettonal ^ittods 
Ctntrai Lscttisn for Reading Matt rlali 
LEA Dir^ter @f All Redding Aativillts 
Report to the Parents 



Avifage Change +,1 2 



TABLi 12 

CRITiRIA JUOaED OP HIGH IMPORTANCE 
ROUND 1 vs. ROUND a COMMON PORULATIQN 



Rank 
Round 1 


Rank 
Round 2 


Item No, 


Mean 


Changt 


14 


IS 


2 


3.91 


-.08 


17.S 


16.S 


16 


3.90 


. +.0S 


17.i 


1&S 


9 


3.90 


+.08 


19 


18 


11 


3.87 


+.03 


15 


20.5 


4 


3,82 


-.09 


21 


20.5 


7 


3.82 


+.07 


22 


20.5 


^9 


a82 


+.14 


20 


20.5 


24 


a82 


+.01 


23 


23 


6 


3,63 


+.13 


25 


24 




3.48 


+.06 


24 


26 


1 


3.40 


-.07 


26 


26 


h 


3.32 


+.09 






Average Change 


+.04 



Peserfptof 



Coord! net ion & Artlgulatlon of All Sptelal Retdlng Fro^ 

grams with the iaile Curfieulum 
Report to the Parinti 
Complite LEA Testing Svitim 
Continuous Staff Diviiapment 
Coordination with Pre school Progfanrii 
Insintives for Staff Devilopment 
Jr.^Sr, High Teaoher Knowledgt of DevelopmintarRead- 

ing 

Materials Ricofnizt Variations In Raee« Cultyre, Sex 

Continyoui Reading Pregfam 

Trained Voluntttr Hilpers 

Report to thi Community 

Adult BmlQ Edygatlon Riadjng Compontnt 



Standard Deviation in Round 2 

The slight increaia In mean value for most of the criteria, while not significant in 
Itself, does indicate a movement toward greater consensus which is significant in the 
standard deviation changes in Round 2. Standard deviations for all Items decreaied in 
Round 2, with an average decrease of ,29, A comparison of the number of Items m each 
of the four standard deviation categories is shown in Table 13, 
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TABLE 13 

STANDARD DEVIATION^ ROUND 1 vs. ROUND 2 
COMMON POPULATION 





SD Cattgory 


Round 
1 


Round 
2 


Ifm No. 




> t.S Grtitiy vdrlablo= 


1 


0 






Quitt vgrlible 


15 


3 


B. 13. 19 




M Snghtly var iable 


10 


23 


ail othifs 




K 3 Oood B^f fffitnt 










Total 


26 


26 





In this round, tht 1 itt m m the greatly variabli Citagory movad into tht quitt 
varlible category and 12 ittmi.moved Into the ilightly vartable catagory. The range of ' 
standard dtviatlons In this round is from 1,05 to A7 (1*57 to ,55 in Round 1), a decrease 
of one full cattgory at the upptr limit (1.57 - 1,05 ^ .52). Thus, tfie standard deviation 
in Round 2 indicates primarily slight differences of opinion among the common popular 
tion about most of the criteria. 

DiscusiiQn 

The Qverall poiltlvf chiinge In means (though In itself insignificant) and standard 
deviations, when taken together, indicate an increase from Round 1 to Round 2 in the 
degree of consensus among the common population of educator respondents in Alabama, 
The lack of significant change in the mians» is was mentioned in the discussion of the 
total population respondents, is not untypical of Delphi studies. The change in standard 
deviations is more common. This change In standard deviation indicates that the re- 
spondents in the common population are less spread apart from the mean in their re- 
sponses. This indicates greater consensus overall among the group for the respective 
criteria, in other words, it could be said that the common group of educatori feel that 
most of the 26 criteria for success are of high or vary high importanct and that there ii 
only slight disagreement about the^ Judgment* 
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V/CONCLUSIQNS 



The Total Population vs« The Common Population 

The total population (all those who reipondtd in each round) was compared to 
tht common population (thoie who responded to both rounds of the surviy) to determine 
if any significant difftrencti exiited between those who answered both rounds and tiiose 
who did not A major question is what difference existed in the two groups in Round % 
since in Round 2 the total and common populations are nearly identicaU 

The data show no significant differences between tfie two groups; the several dif- 
ferences which do exist show the common population to judge the criteria of very slightly 
greater importance than the total population and to have less disagreement about their 
Judgment. These differences are shown by the differences in the means and the standard 
deviations, respectively, for the two groups. To illustrate: 

ROUND I MEANS ^ 



Cefnmoii Pop, Total Pep. 

High Mtan 4.70 4.37 

Low Mean 3.33 3^1S 

Range 1,37 1.19 



The higher high and low means of the common population show the common group to 
place slightly higher importance on the criteria. These differences between the high and low 
means are not equal to even one of the smallest standard deviations^ so they are not significant* 
As far ^ the standard deviations are concerned^ the data show the following: 

ROUND 1 STANDARD DEVIATIONS 



80 Cattgsry Common Pop. Total Pop. 



Griitly variable 1 1 

Quite viriabti 15 19 

Slightly variable 10 6 

Goocf agrefimtnt • ^ 

Total 36 M 



In this round, the common population placed only 16 criteria In the quite variable cate- 
gory compared to 18 for the total population. Fui^ermore« tiie common group pilled 
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only 10 criteria in tht siightly variibli category compired to 6 for total pepulttioa In 
every case, the common population had i lower stindard deviation, although all deviation 
difference were within 0.1 of each other for etch criterion. 

Some intertsting iimllaritles exist between the two groups. Both groups placed 
the same criteria In the same category^ each put half in the extremely high importanct 
category and haif in the high Importance category. While each group did not rank the 
criteria in precisely the same order, they did put them in roughtly the same quartile (top 
half of ixtremely high importance, bottom half of extremely high Importanca, top of 
high importance, and bottom of high Importance). This means that the two groups agree 
about the degree of importance of the.26 criteria. 

Thus, to reiterate: there Is no significant difference in ratings or standard deviations 
between the total and common populations of the Delphi survey, iqually important, 
similarities do exist which show that the two groups essentially see the 26 criteria as of 
high or very high Importance and disagree very little about their judgment* They have also 
ranked the criteria In roughly the same way, placing their strongest support In the criteria 
in the instruction and facilities/materials categories. 

Summary and Discussion 

This Delphi survey, undertaken as part of the Alabama R2R program in the first 
year of its Inception, was Intended to provide Input to the project staff, feedback to the 
respondents, and information to other states undertaking a R2R effort with the possibility 
of duplicating the effort in their state. The instrument consisted of a listing of the 26 
criteria for success developed by the Alabama educatoi^ with a request to rank the Im- 
portance of the 26 criteria on a five-point scale {5 - extremely high Importance, 1 ^ no 
Importance). Two rounds of the survey were completed in the spring of 1974 as repoited 
in the preceding pages. 

The detailed results obtained in Rounds 1 and 2 of the survey are given in earlier 
sections of this report Only a few of the more general observations are discussed here, ^ 
Since there was no signiffcant difference between the total population (all respondent) 

and the common population (the 525 persons who responded to both round! of the 

* 

survey), iMs section will highlight the major survey result, 
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only 10 criteria In the slightly variible catfigory compired to 6 for *t total populitioa In 
every case, the common population had i lower standard deviation^ although all deviation 
difftrtnc^ were within 0.1 of tach other for each criterion. 

Some intertsting iimilaritles exist bitwtin the two groups. Both groups placed 
the same criteria In the same category^ each put half in the extremely high importanct 
category and half in the high Importance category. While each group did not rank the 
criteria in precisely the same order, they did put them in roughtly the same quartile (top 
half of extremely high importance, bottom half of extremely high Importanca, top of 
high importance, and bottom of high Importance). This means that the two groups agree 
about the degree of importance of the.26 criteria. 

Thus, to reiterate* there is no significant difference in ratings or standard deviations 
between the total and common populations of the Delphi survey, iqually important^ 
similarities do exist which show that the two groups essentially see the 26 criteria as of 
high or very high importance and disagree very little about their judgment* They have also 
ranked the criteria in roughly the same way, placing their strongest support in the criteria 
in the instruction and facilities/materials categories* 

Summary and Discussion 

This Delphi survey, undertaken as part of the Alabama R2R program in the first 
year of its Inception, was Intended to provide Input to the project staff, feedback to the 
respondents, and information to other states undertaking a R2R effort with the possibility 
of duplicating the effort In their state. The instrument consisted of a listing of the 26 
criteria for success developed by the Alabama educatoi^ with a request to rank the Im- 
portance of the 26 criteria on a five-point scale {5 - extremely high importance, 1 ^ no 
Importance). Two rounds of the survey were completed in the spring of 1974 as repoited 
In the preceding pages. 

The detailed results obtained in Rounds 1 and 2 of the survey are given in earlier 
sections of this report Only a few of the more general observations are discussed here, ^ 
Since there was no significant difference between the total population (all respondent) 
and the common population (the 625 persons who responded to both roundi of tiit 
survey), iMs section will highlight the major survey rtsult, 
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Implioations of the Study 

Perhaps tht most significant pirt of the study result point to the level of lecep- 
tanat on the part of all iupport groups. By knowing this level and the degree to vvhich 
constnsus was achisvid, the R2R staff knew where m^t of their efforts would need to 
be applied^ In oases where there was a low level of acceptance of crittria by subgroups in 
the state, spicial orientation efforts could be conducted to Increa^ the level. In Instances 
where certain criteria had low acceptance and othem had high acceptance before or after 
Round 2, R2R staff were very much aware of this kind of subgroup opposition. 

Since the development of R2R programs Is based on the criteria and their accept 
tance and operationallzation by all suBgroups, the degree of acceptance takes on a far 
greater Importance in the development process. The accaptarice of the criteria implies the 
need for a massive reorpnization of reading efforts In most school districts. Such efforts 
will require the complete support of all district personneL Realising that even one group 
In a district would actively oppose any of the criteria is an Indication that a critical ele- 
ment of the R2R program may not be Implemented, On the basis of the Delphi data^ the 
R2R staff can better evaluate where its statewide efforts need to be placed. Time and 
staff resources can be effectively and efficiently placed with this type of Information. 

Another significant result of the study concerns me^urlng the effectiveness of 
input. That Is, when intervention and orientation strategies are used to educate subgroups 
about criteria which they oppose, the degree to which their attitudes and opinions change 
can be accurately assessed through the continued use of the Instrument through a third 
round for those criteria about which there is concern. 

The last and most significant aspect of the study concerns the ability of the instru- 
ment to provide an overall perspective statewide for the development of a major reading 
program. From this pe^pectlve, an entira reading effort can be faunched with a knowledge 
of the degree of acceptance of i^ major components as well ai where the major obstacles 
would occur. Such a perspective must be eKtremely valuable to any R2R itate^s planning 
and development effort. 
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Appindix Dilphi Surviy li^trument 
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ERIC 



ALABAMA RIQHT TO READ PROaRAM 
ALABAMA STATi DiPARTMlNT OF EDUCATION 



Dtar Collaaguii 

A few weeks ago you riceivtd a letter requiting you to comp!iti a qgestionnairt re^fding criteria for a 
su€€mful rtidlng progi^m. The risponse to that queitfonniire vm eistrimily f^orablt, and wi ippre^lata ths timo 
you ^nt In working on It. 

As we Indlcitid In thm Initial set of doauminti^ all reiponsgi have been analyzed, and wt are now ready to 
begin the ieeond phase of the study. The orlglnil qyeftlonnairt hm been rnodlfled to Inslude aver^ re^nie 
from individuals In your group for each of thi eriterla. With thii svmgs whieh you will find printed on ti^e qu^tion- 
nalre, we would like you to repeat the original proeea and mluatt the eri^rla a second time* 

In order for this study to be valid. It Is critl^l that you eomplett the qu^tionnairt again. We feel that tfit 
time yog spend in completing the qutstlonnaife will N rewarded by providing Information for Alabania to devel^ 
one of the finest Right to Read pr^rams in the ^untry. 

Your cooperatfon in this ^fort Is gi^atfy ap^teiated. Please return the questlonnafrt before April 2S» 1974. 

Rtchard Mcirlde, Direstor 
Af^ama Right to Read Pr^tmm 



ALABAMA RIGHT TO READ 
DiLPHrQUESTIQNNAIRl INFORMATION 

Plem return questionnaire by April 2i, 1974 \ 

Fill In or complete the following 

•L Last Name FInt Initial 

2. School 

3. School Address - . 

4. School District , _ _. 

Circle the code number below that prtcidas the category of reipondeno to vvhlch you belong: lug., 4.1, Supirlntendtntt 
or Assistant Superintendent. 



Coda Numbers 


Category 


4.1 


Superlntindent or Anistant Superintendent 


4.2 


ilementary Princlpil 


4,3 


MIddte or Junior High Principal 


4.4 


High School Principal 


4.e 


Local Education Agency Supervisor m PIrtctor of Instnjction 


4J 


Current Local ichoot District Right to Read Director 


4.7 


f limentary Teacher 


4.8 


Middle or Junior High Teacher 


4,9 


High School Teacher 


4,10 


Higher iducation Reading instructor 


4.11 


State Rl#it to Read Advisory Commlulon 


412 


State Right to Rf^ Tmk Force 


4.13 


SDi Consultant 


4.14 





SpMify 



*AII infermttlan provWtd by tha rt^ndtnn It cenfMfntlal. 
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ALABAMA RIGHT TO READ 
DELPHI QUESTIONNAIRE 



Pftase glfcli tht numbff of the categoFits btlew that srn most el©i«ly rilittd to your judgmtnt of tht impofwce of 
esgh criterion. 

s * 

Arrsy of Crittrig for i Succt^ful Riadmg i 1 

ft ^ 0 

Progrsm (addittonat can be listfd on V | E J 

final i i ^ i 



i S 1 1 



6 @ Q 

-ISO 

2 1 0 



1i iach LiA teaeher of fisdlng uses a scope ind itquinei 4p3 S 4 3 2 1 0 

of learninp dtslgned to iniure the Mqulsltlon of ' ^ 

riidlng skitii, 

2. Thirt ii coordination and artlculition between gll 3,S .5 4 3 210 
federally funded, volunteer, arid other reading and 

languige arts progranis with the bmic rtadmg cyrrlcutum. 

3. A continuous recordkieplng system of reading progrea 4.2 i 4 3 2 1 0 
is maintained for each Indiylduil student 

4. Tht Lf A works cooptratively with eictiting pre>ichpol 3 J § 4 3 2 1 0 
components to coordtngte and articulati regdlng pro^ 

grams. ^ ^ 

5. The Li A has an adult basic education reading compontnt^ 3.0 5 4 3 2\ 1 0 

§. The LiA has a continuous educitlonal program which irt^ 3.6 1 4 3 2 1 0 

eludes provlilon for symmer Instruction In reading. 

7. Thi board of education of the Lt A has an Incentive 3,6 5 4 3 2 1 0 
program for teacher staff developmint In reading. 

8. Each school In the LEA has a media ctnter which Is 4.0 5 4 3 2 1 0 
operated on an open balls and is readily accessible to 

students and teachers. 

9^ The LEA has complete testing system which Includes 3.7 S 4 3 2 1 0 

the use of criterion-referenced measures. 

10. The LEA varies the studint/teacherratiQi as necessary 4,2 5 4 3 2 1 0 
to meet the objectives of the reidlng Instructional pr^ 

gram^ 

11. The LiA has § continuous staff development program 3.7 5 4 3 2 1 0 
In reading for ait ttachers, administrators, and sup- 

portlve person neli ^ 

12. The LEA provldei trilntng to teachers In the content 4«2 5 4 3 2 1 0 
areas to develop competence vvhich will allow them to 

adjust Instruction to the valuing reidlng achievement 
ievels of their students^ 

13. The LEA has trained voluntetr heipers in reading S.g 5 4 3 2 1 0 
instruction. 

14. Each school In the LEA has a media center which It. 3J 6 4 3 2 1 0 
Itafftd by proft»iofial and njppdrtlvt pa r^nnvL 
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16^ Tha Li A a dlrtetor of ill reading aetlvititi who hm 
the authority, responitbill^, and timi granttd by the 
SJptrintendint and board of tducatfoft to organiie, im- 
, plamenf and eoordinata g asmprihenjlve re idlng progrgnL 

1 6. iaeh LEA hm m r^or ting lyi tern that f yliy, accyfatgiy afid 
ipecifioilly communleatn § student's progri^ in riadlng 
to parens. 

17* Tiit LEA introdues, explains, and ptriedleilly reportt 
the rtadlng program to the school eemmunlty^ 

18^ The Lt A shows winingniss to share instrugtlonil 

methods and fnaterials whrgh hive proved ef festive In 
reading programs. 

19, Teach irs of reading at thi Junior and senior high school 
levels have a demonstrsttd knowledge of devilopmentil 
rtiding m It relates to the reading eurrleylum of the Lf A* 

20, Frovlsions art made for ttichlng evgiv student it hte 
own instry^tional level and Itirnlng rate. 

21* The ttiching^learning environment li aondudfve to the 
devel^ment of positive attitude toward rtiding. 

22* iviiy teicher demonstrits a knowltdgi of various 

methods and techniquts used In the ttashlng of rtiding 
to make provision for the differences that exist among 
student. 

23. Appropriate suppltmentary reading mat^lals to support 
the b^ig reading eurrlculum are provided and utiliied. 

24« Mattriali are utilized which rtoognize dlfftrtnt rac^ 
cultures, and sexts. 

21, A wide virltty of sypportlvt media on ill levils of 
learning is gvailabte and readily accessible. 

2t» Initruetionil and practlee reading miterlals ve flM 
in i central location in ea^ school for use by all 
teachtrsaintidid. 
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ROUND 1 

CRITIRIA RANK, MIAN, AND STANDARD DgVIATiON 
TOTAL POFULATION 



Rank 



\M 
3 

4.S 
€ 

7.S 
7.5 

9 
10 
11 

12 
13 
14 

15 

1&S 

163 

18 

19 

20 

21 

22 

23 

24 

25 

26 



20 
21 
22 
1 

23 
10 
12 

3 
25 

8 
18 
15 
26 

2 
14 
16 

4 
11 

9 
24 

7 
19 

6 
17 
13 
S 



^Importance. 
4 to 5 
3 to 3.99 
2 to 2.99 
1 to 1.99 

^0tslN*919. 



txtrtmify high 
high 
ffiidlym 
lovv 



Mian* 



4.62 

4.62 

4,37 

4.35 

4.35 

4,26 

4.21 

4.21 

4.15 

4.11 

4.08 

4.07 

4.02 

3MB 

3.83 

3.82 

3.82 

3.81 

3.79 

3.73 

3.69 

3.61 

ase 

3.42 

440 

3.1S 



Stindard 
Divlatlaii 



0.85 

0.70 

0.96 

1,03 

0.79 

1.08 

1,09 

1.00 

0,95 

1.18 

1.00 

1.21 

Ml 

1,26 

1.25 

1,15 

1.23 

1,22 

1.25 

1,22 

1.31 

1.60 

1.31 

1.16 

1.35 

1.45 



896 



88i 
880 
887 
894 
893 
895 

Bm 

896 
890 
889 
884 
889 
890 
89S 
889 
894 
891 
884 
889 
894 
893 
896 
893 
^0 
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ROUND 2 

CRITiRIA RANK, MfAN, AND STANDARD DEVIATION 
TOTAL POPULATION 



Rank 


Ofiterloft 
Nq, 


Mian* 


Standard 




1 


20 


^ 4.74 


0.62 


595 


2 


21 


4.73 


0.56 


§90 


3 


22 


4.62 


0.60 


590 


4 


23 


4.BS 


0.64 


689 


i 


1 


4.46 


0.74 


587 


6 


10 


4.42 


0.73 


_ . _ 590 


7 


3 


4.34 


0.76 


819 


8 


25 


4J3 


OM 


687 


9 


12 


4.30 


0.80 


592 


10 


8 


4.29 


0.75 


690 


11 


IS 


4.17 


0.68 


592 


12 


15 


4.16 


0,82 


684 


13 


26 


4.14 


0.78 


590 


14 


14 


4.00 


0.83 


692 


IB 


2 


3.i1 


0.86 


687 


16 


9 


3.90 


0J4 


537 


17.5 


16 


3.87 


0J3 


693 


17.S 


11 


3.87 


0.90 


591 


19 


4 


3.82 


0.85 


582 


20 


19 


3.81 


1.08 


696 


21 


7 


3.80 . ^ ^^ .^ ^ 




— 688 


22 


24 


3.78 


0.97 


589 


23 


6 


3*87 


0.84 


687 


24 


13 


3.45 


1.03 


691 


2i 


17 


3.39 


0.92 


593 


26 


S 


3.34 


1.00 


684 



note a for Round 1, total papulition, 
^ottl N - 199, 
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ROUND 1 

CRITERIA RANK, MIAN, AND STANDARD DEVIATION 
COMMON POPULATION 





Criteridn 




iSUndard 




No. 




Df vlitlon 




1.S 


21 


#^ 4,70 




522 


1.S 


20 


4.70 




521 


3 


1 


4.43 




516 


4 


22 


4.41 


OA 


521 


5 


23 


4.37 


«/3 


521 


6 


10 


4.32 




522 


7 


12 


4.28 




520 


8 


3 


4.22 






9 


8 


4.18 


1.08 


522 


10 


25 


4.17 


,ii 


119 


11 


18 


4.13 


.95 


120 


12 


26 


4.01 


1.03 


519 


13 


1i 


4,07 


1,22 


119 


14 


2 


3Ji 


1.13 


523 


15 


4 


3.91 


1.12 


521 


16 


14 


3.89 


1,14 


520 


17.5 


9' 


3JS 


1.21 


522 


17.5 


16 


3.85 


1,07 


522 


10 


11 


3.84 


1.15 


522 


20 


24 


3,77 


1,17 


519 


21 


7 


3Ji 


1.21 


519 


22 


19 


3J8 


1,S7 


522 


23 


6 


3JS 


1.28 


S19 


24 


17 


3,47 


1.08 


523 


25 


13 


3.^ 


1,31 


521 


28 


6 


3.^ 


1,41 


520 



note ■ for Round 1. tout popyinlon. 
^etal N ■ KB. 
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ROUNDS 

CRITERIA RANK, MIAN. AND STANDARD DEVIATION 
aOMMON POPULATION 









Stindird 




Rank 




Mi in* 


Ptviat lOEi 






^1 


" 4.77 


0.47 


616 


1 S 

l>B 


dU 




0.5i 


521 




dd 


4. €3 


0^61 


516 






4,qQ 


0.58 


i16 




1 


4,47 


0.67 


515 




lU 


4,4S 


0.66 


516 


* 


da 


4.97 


0.58 


513 


8 


3 


4.33 


0.76 


515 


9 


12 


4.32 


0.75 


518 


10 


S 


4.28 


0,76 


516 


11 


18 


4.19 


0.66 


520 


12 


2i 


4.17 


0,75 


B16 » 


13 


15 


4.16 


0.82 


512 


14 


14 


4.00 


0,83 


513 


IS 


2 


3.91 


0J6 


515 


163 * 


16 


3.90 


0.80 


519 


las 


§ 


3.90 


0.79 


514 


18 


11 


3.87 


0.87 


617 


20.5 


4 


3.82 


0.85 


517 


20.5 


7 


3.82 


0.91 


514 • 


20.5 


19 


3.82 


I.Oi 


522 


20.5 


24 


3.82 


.96 


515 


23 


6 


3.68 


.91 


514 


24 


13 


3.48 


too 


518 


25 


17 


3.40 


.90 


511 


26 


i 


3.33 


1.01 


511 



^S^note a for Round 1, tpfel populitidni 
^otsl N * 525. 
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CHANGE IN MiAN 
TOTAL POPULATION vs. COMMON POPULATION 



Total Pepuiitliin 



Orlttrlon 
No. 



Common Pspytitlon 



Round 
1 



Round 
2 



Ohanga 



Round 
1 



1 
2 
3 
4 
9 
6 
7 
8 
9 
10 
11 
12 
13 
14 
15 
16 
17 
18 
19 
20 
21 
22 
23 
24 
25 
26 



4,35 

asi 

4.21 

a82 
ai8 

ase 
a69 

4.11 

3.79 
426 

asi 

4.21 
3.40 
3.83 
4,07 
aS2 
3.42 
4.08 
3.51 
4.62 
4.62 
4,37 
4.3S 

a73 

4.15 
4.02 



4.46 

3.91 

4.34 

3,82 

3.34 

3.67 

3.79 

4.29 

3.90 

4.42 

3.87 

4.30 

3.45 

4,00 

4.1 i 

3,87 

3.39 
4.17 
3.81 
4.74 
4.73 
4.62 
4.58 
a78 
4*33 
4.14 

Averaga Chingt 



+,11 

+,05 

+.13 

^,00 

+.15 

+.11 

+.11 

+.18 

+.11 

+.16 

+.06 

+.08 

+.06 

+.17 

+.09 

+.05 

«.03 

+.09 

+.20 

+*12 

+.12 

+.25 

+.23 

+.05 

+.19 

+.11 

+.11 



4,43 
3.99 
4,22 
3.91 
3,23 
3.S5 

a7B 

4.18 
3.85 
4.32 
3.84 
4.26 
3.43 
3.89 
4.07 
3.85 
3.47 
C13 

aes 

4.69 
4.70 
4.41 
4.37 

3.77 
4.17 
4.08 



Round 
2 



4.47 

aoi 

4.33 
3.82 
3.33 

a68 

3.82 
4.28 

a88 

4.45 

3,87 

4.32 

3,48 , 

4.00 

4.16 

3,90 

3.40 

4.19 

3.82 

4.77 

4,78 

4.63 

4.60 

a82 

4.37 

4.17 



+.04 

-.07 

+.11 

-.09 

+,10 

+.13 

+.07 

+,10 

+.05 

+,12 

+.03 

+,06 

+.04 

+,11 

+.09 

+.05 

-.08 

+.06 

+.14' 

+.09 

+.08 

+,22 

+.23 

+,05 

+.19 

+..08 



Avtragt Ghangi +.10 



41 



46 



CHANGE IN STANDARD DIVIATION 
TOTAL POPULATION vs. COMMON POPULATION 



Totil Population 



No. 



Common Pepulition 



Round 
1 



Round 
2 



ChanH 



Round 
1 



Round 
2 



1 
2 
3 
4 
i 
6 
7 
8 
9 
10 
11 
12 
13 
14 
15 
16 
17 
18 
10 
20 
21 
22 
23 
24 
25 
26 



1,03 

1,23 

1.00 

1,23 

1,45 

1.31 

1.31 

1,18 

1.25 

1.03 

1,22 

t09 

1.35 

1.25 

1.21 

1,15 

1.16 

1.00 

1.60 

0.85 

OJO 

0.96 

a79 

1.22 

0.95 

1.11 



Change 



0.74 

0.86 

0,76 

0.85 

1.00 

0,84 

0J2 

0.75 

0.84 

0,73 

0.90 

0,80 

t03 

0.83 

0.82 

0.83 

0,92 

0.68 

1.08 

0,61 

0.56 

0.60 

0.64 

0,97 

0,64 

0.78 

Avera^ Changt 



#^.29 
-.40 
-,24 
-,38 
-,45 
-,47 
-,3i 

-,43 

-.41 

-,35 

-,33 

-.29 

-J2 

-,42 

-.40 

-.32 

-.24 

-,32 

-,52 

-.24 

-.14 

-,38 

-.15 

-,25 

-.31 

-.33 

-.34 



OJi 

1,13 

0.98 

1,12 

1,41 

1.26 

1,21 

1.08 

1,21 

0.99 

1.15 

0,99 

1.31 

1,14 

1.22 

1,07 

1,08 

0.95 

1,57 

0.73 

0.56 

0,94 

0.75 

1,17 

0.91 

1,03 



0,67 , 

0.86 

0,76 

0.8S 

1.01 

0,91 

0.91 

0,76 

0,79 

0,66 

0.87 

0.75 V ^ 

1.00 

0.83 

0,82 

0,79 

0,90 

0,68 

1,05 

0,55 

0.47 

0.61 

0,58 

0,96 

0.53 

0,75 

Averagi Changi 



-.23 
-,27 
-.22 
-,27 
-.39 
-.36 
-.31 
-.32 
-.42 
-,33 
-.27 
-.24 
-.31 
-,32 
-.40 
-,28 
-,19 
-,29 
-.52 
-.18 
-.09 
-.33 
-,17 
-,22 
-.32 
-.28 
-.29 
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Appt ndix Ct Individual Statistics 
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TOTAL POPULATION MIAN RESPONSE 
ROUND 1 



' RiipontoiSreup 



No. 


N 


1 


N 


1 


N 


3 


H 


4 N i 


N 




H 


7 


N 


§ 


N 9 


N 10 


H 


It 


N 12 N 13 fi .14* 


1 


41 


4.56 


32 


4,47 


18 


m 


21 


4,1? 29 4,76 8 4,17 


360 


4,34 


116 4,43 


111 4,07 


13 m 


4 


4.10 


3 4,00 8 4,37 72 m 


2 


41 


4,33 


33 


4.09 


17 


m 


21 


4.19 29 4.41 


i 


4.62 


303 


3.7? 


117 


a72 


111 3.62 


13 4.48 4 4.60 4 4.?6 8 3,87 76 4IS 


3 


4B 


4.33 


33 


4,41 


18 


i17 


21 


4,43 29 4,79 


8 


4,87 


365 


4.18 


117 


4,11 


113 3,96 


13 4,69 


i 


4,00 


4 4.00 8 4.7S n m. 


4 


44 


m 


33' 3.g? 


18 


3,72 


21 


3,62 29 4.24 


8 


4,10 


363 


3.76 


118 


m 


151 3.75 


13 4.46 


4 


4.50 


4 4.2E 8 4.28 71 m 


S 


.41 


3,2? 


31 


3,13 


18 


3.08 


21 


2.S1 29 3.72 


6 


4.12 


3S4 


3,08 


116 


3,19 


113 3.19 


13 3.69 


4 


3,50 


4 4.00 8 3,00 78 3.21 


§ 


'44 


348 


33 


3.11 


18 


3,10 


21 


3,24 29 3.72 


a 


3.71 


361 


3,§8 


118 


3,64 


153 111 


13 3.92 


4 


3.00 


4 3,50 8 3.62 71 3.^ 


7 


4i 


317 


33 


3.11 


18 


3.iS 


21 


3.12 28 ♦3.78 


8 


4,62 


362 


3,§2 


lie 


3,70 110 3.69 '13 4.00 5 3,40 


4 3.71 8 4,00 76 3.71 


1 


40 


4,ie 


33 


4.33 


11 


3.44 


21 


4,14 29 4.82 


8 


4.50 


361 


4,04 


119 


4,13 


113 4,67 


13 4.46 


4 


3.00 


4 4,25 8 4.25 78 45l 


9 


4i 


3.93 


33 


4.09 


18 


4,00 


21 


4.14 2i 4.34 


8 


4.62 


362 


3.67 


119 


3,68 


151 3.72 


12 4.33 


1 


3,80 


4 4,25 8 4.28 76 3.SS 


10 


41 


4.07 


33 


4.4B 


ie 


m 


21 


3.§S 29 4.31 


8 


4.87 


313 


4,22 llli 


4.M 


112 4.28 


13 4,33 


6 


3,80 


4 4.i0 § 4.10 71 4,U 


11 


41 


m 


34 


4.41 


18 


3.89 


20 


3.§i 2i 4,34 


8 


4,10 


368 


3.?3 


118 


3.60 


182 3,82 


13 4,54 


I 


3.40 


4 3.7S 7 4,71 ?l 4.0S 


12 


45 


4,31 


34 


4.10 


11 


4,28 


20 


3,71 29 4.31 


8 


4,87 


385 


4,21 


118 


4,11 


112 4,12 


12 m 


6 


3.10 


4 5.60 8 4.21 71 4.^ 


13 


44 


2.01 


34 


3.00^ 


18 


3.28 


20 


3.0e 21 3.41 


a 


3.87 


364 


3,47 


lie 


3.55 


111 3.26 


13 3.61 


1 


3.20 


4 4,25 8 3,37 76 3,22 


14 


4B 


4,00 


34 


4.M 


17 


3.71 


20 


4.1 29 4.24 




3.87 


364 


3.76 


116 


3,93 


151 3,87 


13 3,77 


i 


3.80 


4 3.71 8 4,21 76 313 


IS 


41 


4.11 


33 


4.24 


18 


4.17 


20 


4,4§ 21 4,34 


8 


4.3? 


384 


3,i1 


113 




110 4.01 


13 4.11 


i 


4.40 


4 4.75 a 4,37 75 '4,48 


16 


44 


3.01 


34 


4.21 


18 


3,78 


p 4.01 29 4,24 


a 


4.82 


366 


3,75 


119 


3.81 


152 3.64 


13 4,31 


4 


4.00 


4 4.71 8 4,21 76 3JI 


17 


45 


3.62 


34 


3.76 


11 


3.D6 


20 3,70 29 3,97 8 


4.37 


311 


3.26 


Hi 


3.39 


112 3.18 


13 4.11 


i 


4,40 


4 4,71 % m n m 


IS 


45 


3.73 


34 


4.2i 


18 


3,§4 


20 


4.15 29 4J1 


8 


4.2i 


363 


4.01 


117 


4,06 


ill 4,07 


13 4,38 


i 


4.20 


4 4,21 8 4.S0 71 4.17 


19 


41 


310 


34 


3JI 


18 


3,81 


20 


3,95 29 4.17 


a 


4,62 


363 


3,19 


119 


4,04 


112 3,74 


13 4.77 


i 


3.80 


4 4.71 8 4.37 76 m 


20 


45 


4.50 


34 


4,53 


18 


4.g0 


20 


4.61 21 4,89 


8 


4,7i 


367 


4,^ 


113 


4.44 


111 4.53 


13 0.00 


i 


4,80 


4 5.00 1 417 76 4,60 


21 


4S 


4.10 


34 


4,62 


IS 


4.39 


20 


4,80 29 4,83 


8 


4.7i 


361 


4.S7 


119 


4,M 


151 4,41 


13 4.92 


i 


4.80 


4 1.00 8 4,8? 76 4,11 


22 


44 


m 


33 


4.i4 


17 


4.29 


20 


4.10 29 4.62 


8 


4.75 


364 


4,48 


117 


4,07 


110 4,17 


12 4,50 


i 


4,40 


4 S.00 8 4,37 71 441 


23 


44 


4.41 


33 


4.31 


17 


4,29 


20 


4,31 29 4,48 


8 


4.62 


361 


4.36 


11? 


4,26 


150 4,30 


12 4.63 


1 


4.20 


4 4.71 8 4.10 76 4a 


24 


41 


m 


33 


3.ii 


17 


3.e§ 


20 


3,31 29 4.1? 


8 


4,50 


314 


3.67 


114 


3JQ 


'no 3.64 


12 4.17 


i 


3.00 


4 3,00 a 3,37 75 3.93 


2§ 


40 




33 


4,24 


16 


4.00 


20 


4,05 21 4,41 


3 


4,26 


362 


4.18 


117 


3.97 


150 m 


11 4,33 


i 


4.20 


4 4.75 i 4.21 7§ m 


26 


40 


353 


33 


4.06 


17 


3.71 


20 


ae§ 29 4,34 


8 


4.50 


363 


4,05 


116 


3,92 


149 3.99 


12 4.08 


6 


3.S0 


4 4,10 8 4,25 7§ 4.13 


Total N 


Ah 




18 


2 


1 . 29 




8 


3J 


li 


11 


§ 


113 


13 




i 


4 8. .. 76 



TOTAL POPULATION STANDARD DIVIATION 
ROUND 1 




liii 


1 


m 
1 


• 


4 


§ 


jt 

8 


1 


3 


i 


10 


11 


11 


11 


14 


1 


0.588 


0J61 


0.712 


0,7^6 


0.431 


0,354 


1.038 


P,90i 


1.367 


0,480 


S.I771 


1,000 


0,744 


1.032 


2 


0174 


1.121 


0.870 


0.749 


0,632 


P.il7 


1.247 


1.421 


1,491 


0.660 


1.166 


0,600 


1,642 


1.143 


3 


0,769 


o.es7 


0,114 


0,676 


0.412 


0.314 


0.991 


1,016 


1,297 


0,480 


0.707 


0.818 


0,463 


0.858 


4 


1.216 


I.04S 


1,227 


1.161 


OjiS 


0.766 


1,212 


1.311 


1.466 


0.60D 


0,677 


0,917 


0,707 


1.073 


1 


1J7E 


1.403 


iOii 


1.123 


0.922 


0,834 


1.603 


1,434 


1,432 


0,751 


1.000 


0.816 


1.512 


1.334 


6 


1.110 


1.149 


l.Mi 


1,333 


0,922 


0.886 


1.393 


1.350 


' 1,367 


1.P38 


0,116 


1,291 


0,744 


1,256 


7 


0.921 


1.331 


1.097 


1.101 


0.711 


0,744 


1.394 


1.348 


1.331 


1,165 


0.894 


o.ei7 


0,026 


1,307 


8 


mi 


0,736 


I.3S1 


0,793 


0,622 


0.134 


U84 


1,214 


1,273 


0,519 


1.414 


0,907 


0,836 


0.974 


1 ' 


\ 0.941 


0179 


0.M7 


0.910 


0,060 


0,517 


1,263 


1,478 


1.327 


0.061 


0.837 


0.600 


ae36^ 


1,25) 


10 


' Oli§ 


0,711 


0.72i 


1.244 


a7ai 


0.314 


1,233 


1.027 


1,033 


0,768 


1.0^ 


1000 


0,634 


0.946 


ii 


0.793 


0.701 


0.132 


0.326 


0.769 


0.716 


1.213 


1,241 


1,311 


a877 


2.074 


1.2i 


0.483 


1,133 


12 


0.733 


0.707 


0.712 


1.164 


o.iao 


0.354 


1.1B1 


1,273 


1,173 


0.116 


im 


0.00 


1,035 


0,859 


13 


1.062 


1.121 


1.074 


1.356 


1.061 


1.126 


1,371 


1,339 


1,561 


0,660 


''0.837 


0.967 


0.617 


1,302 


14 


0,879 


0.830 


1,160 


0,908 


0.372 


1.727 


1.342 


1.231 


1.355 


1,301 


0.337 


1,600 


0.707 


1,131 


1$ 


Plii 


1.062 


0,923 


0.837 


1.009 


1.768 


1,M7 


1,248 


1,170 


0.17 


0.894 


m 


0.617 


0.795 


16 


0.802 


Q.iO 


O.B7f 

m 


0,044 


0.689 


0,517 


1.207 


1.262^ 


1,205 


0.630 


1.165 


0,500 


0.463 


1,178 


17 


016 


m 


, 0,923 


0.826 


0.517 


1.221 




'1;137- 




0.S94 


0.600 


0.707 


0.36i 


18 


0.789 


0.123 


0.726 


0.171 


0.604 


0,111 


1,064 


1.019 


1.043 


0,961 


1.304 


0,95? 


0.534 


0121 


19 


1.140 


1121 


0.003 


1.009 


1.136 


0.744 


1,870 


1,W2 


,1.493 


0.199 


0.337 


0,500 


0117 


1.354 


20 
*1 


a6i7 
0.491 


1.187 
0107 


0.867 
0.603 


0.107 
0,523 


m 

0J4 


0,707 
0,707 


0.753 
0.626 


i.toa 

0.756 


0,937 
0.082 


0.0 
0.277 


0.447 
0,143 


0.0 
0.0 


y54 
0.354 


0,865 
0,612 


22 


0,313 


1.003 


0.^6 


1.209 


0,622 


0,461 


0.816 


1,179 


1,067 


1.441 


0.894 


0.0 ■ 


0.617 


0,741 




0.622 


0.904 


o,oas 


0,671 


0,574 


0,744 


0.719 


0.822 


0.826 


0.616 


0,337 


0.600 


0,534 


0197 


24 


1.217 


1.023 


0.712 


1.^01 


0.760 


0.716 


1,307 


1.119' 


1.233 


0,335 


0.707 


0.316 


1.113 


1,107 


n 


0.141 


0.701 


0,804 


0,759 


0.733 


1,165 


0,N2 


1.141 


0.976 


0,661 


0.837 


0,500 


1.035 


0.815 




0.930 


0,827 


1.106 


1.191 


0.721 


0.766 


, 1,125 


1.279 


1.171 


0.i1S 




0.677 


1.036 


1.004 



TOTAL POFULATION MIANRESPONIi 
mD2 



^ft N I N I N 3 N 4 N i N 8 N k ?: N r 12 H M H M 

1 31 m 21 418 IB MO 14 4|?1 24 4,833 1 i,00 m 4,3?r ?Q 4,41? :10ir4,3Da: 14: 4,714 m t 4.00 9 4.8^' 46 4.S0l| 

2 M 4.400 21 4.32 10 m 14 4.288 24 4.i42 ?, 4.B71 218 3,70r70 3,i«: 108[ 3,657 14 4,171 :S IQ^ 5.00 9 4,111 4e 4.174-I 

3 3S m 25 4.68 10 4.40 14 4,B71 24 4175 7 §,00 210 4,281 71 4.333 1M' 3.01114^.1^^^ i mi 46 '4.391^| 

4 31 4.000 25 3,i 10 3.70 13 3.815 24 4,208 7 4.571 218 3,176 70 3.7t 108 1722 K 4.75 2 1.00 8 4.333 41 iM"^ 

I 35 3.343 25 3,04 10 3.30 14 2,317 24 3.833 7 4,288 211; 3,20 71 3,324: !M 3.3i 14 3714 j4 5,00 1 3.887 4i^ 3.2St | 

6 35 3J14 2i 318 10 3,10 14 3171 U 3.833 7 4,216 217 3,064 71 3,69S' 109 :3i7p'l4 418 4 3,21 2 3.60 1 3778 46' 3.522^ 

7 3i 3.el4 2S 4.12 10 3.80 14 3,143 2r3.D&8 7 4.157 218 3.803 71 3732 i 1,50 9 4.111 '46 3,Kli| 

8 3S 4.057 2S 4.GQ 10 3,70 14 4.307 24 4,620 7 4.571 219 4.247 71 4.m 109 :4;22i 14 4.871 J 3.00 2 5,00:0 4.444 48 ^4.4071 
i 35 4,171 JI 4.32 10 3,90 14 4.288 24 4.333 7 4.429 ^219 3.753 70 3.814 IDS : 3,750 14 4,317 4^^^^3 i.00 9 4,444 ■ 48" 3j35'f 
10 35 4.314 25 4.44 10 4,30 11 4.00 24 4.458 7 4157 218 4.404 71 4,338 1M 4,422 14 , 4714 I 4,291 4.50 9 4.817 48 4.130 ^| 

II 31 4.314 24 4,417 9 3107 14 3.671 24 4.458 7 4,714 221 3756 71 3,813: 109 3132^^^ 3,50 9 ' 4.558 

12 35 057 24 4.628 10 4.10 15 3.86? 24 4418, 7 4,571 221 4.287 71 4.206^109 : 4.23^ 14 4.643 4 4 21 2 2,50 9 4.333 47 1.468^ 

13 38 3.QQQ 2fftl83 10 3.00 14 2,786 24 3.542 7 4429 220 3.591 72 318 109 3,201 13:4^ 4 3.71 2 2.10 9 3556 '47 '3.298 (| 

14 38 4,081 23 4.391 10 3,80 14 4143 24 4,333 7 4.288 221 3137 72 3.819 :109 3,938: 14 4.143 4 3,10 2 4,00 i 4.333 47 1213 

15 34 4,019 23 4,217 10 3,90 12 4,333 24 4.417 7 4.571 220 4.018 71 4.141:109 4,009 14 4,429 4 4.75 2 4.S0 8 4.50 46 4522^ 

16 38 3.91? 24 4.375 10 3.10 14 3,857 24 4.375 7 4,429 221 3.105 72 3,?92 109 3,6« 4,50 9 4.444 47 37663 

17 35 3.057 24 3,792 10 3.30 IS 3.533 24 4.187 7 4.143 222 3.230 72 3.208 108 3.231 14 4,214 4 4.25 2 4.50 9 4522 47 3.277'','f 
'18 3i 3,80? 24 4,25 10 4.10 15 4.133 24 4.208 7 4,143 221 4,183 72 4.202 109 4.110 14 4.216 4 4,00 2 4,10 8 4.625 4? 4.213. J 

19 38 4.083 24 4.00 10 3.70 11% 24 4.208 7 4.429 223 3.395 7 2 4.083 109 3.972 14 4.929 4 ■ 4.00^ 2 4.50 9 4.444 ' 47 ' 3.809^1 

20 36 4.722 24 4.833 10 4.40 14 5.00 24 iOO ? 4.857 223 4.78? 72 4,M9 109 4,615 14 100 4 4.75 2 m 9 1000 47 m.£ 

21 35 im 25 4.80 10 4.40 14 1.00 24 4,875 7 4.714 222 4.7i ?2 4.708 106 4.613 13 4.846 4 4.71 2 4,10 9 4.889 47 . 4.766^^ 

22 35 4,514 25 4,72 10 4,40 14 4.571 24 4.710 7 4,157 220 4.741 72 4.417 107 4.411 13 4.148 8 4,40 2 5.00 9 4.667 47 4.74i':| 

23 35 4.606 25 4,16 10 4,30 14 4.429 24 4.833 7 417 221 4.079 7 2 4.50 107 4,105 13 4,692 4 4.50 2 i.00 8 4,7B0. 47 

24 35 3.B71 25 3.63 10 3,60 15 3.40 24 4.292 7 4714 219 3.648 72 MOT 107 3.785 13 418 4 3,25 2 3.B0 9 3.556 47 4.128".| 

25 34 4.235 25 4.28 10 4.10 14 4.143 24 4.B83 7 4,571 219 4.324 72 4.278 107 . 4.299 13 4.S38 4 4.00 2 5,00 9 4,444 47 4|ir'l 

26 35 3.800 2B 4,40 10 3.80 14 3,171 24 4.417 7 4,171 221 4,222 72 4,00 107 4.075 13 4.00 4 4.00 2 4,00 9 4.222 47 4577 :| 
Tetil N 30 25 10 15 -^-"^'^'^y^^^^—^— ™— ^.^-^^ 



TOTAL POPULATION STANDARD DiVIATiON 
ROUND 2 
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1*1 
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0,4011 


O.B§i8 


0,1164 


0,6136 


0J807 


0,0 


0,7113 


0.7128 


0,1413 


0,4633 


0.9174 


0.0 


0.3333 


0.6912 


2 


0.§531 


0,7483 


0.6325 


0,6112 


0,1882 


0,6345 


0,8248 
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0.9683 
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0,0 


ao 
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0,5693 
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0,6272 


0,5104 
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0,72ii 


0.6312 
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0.1781 
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0.8002 
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0.0 
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0,8321 


1.1209 


0.1493 


0.5136 


0,7020 


0J55i 


0.0776 


0,8716 


0.9531 


0.3254 


0,500 


0,7071 


0.4410 


0,7525 
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DATA ANALYSIS 



All dita analyits perFormid for the Delphi study wis dont on an IBM 370/168 
eomputir. The mtani, standard deviations, and sannple sizes (N) wire obtalntd from a 
statistical summary (STSUM) program which is part of Penn State's statistiGal paekagt 
(STPAC), 

STPAC is a collection of statistical routines that his ivolvid with the dtvt lopment 
of Penn State's Computation Center. The programs originated from i variety of sources; 
however, they have all been modified so that they run under a single control program using 
standardized control instructions andjnput conventions. 

STSUM is a FORTRAN IV program written by a member of the computation ceriter 
staff. It is designed to be used m an independent processing program for computing sum- 
mary statistics. Data input to STSUM is a matrix In which the columns represent variables 
or treatments, and the rows represent the observational units. In our data analysis, we 
have 26 variables, repreienting the 26 items on the questionnaire. The number of treaMents 
or observations is the number of respondents. These observations vary in number for 
various groups, but in the common population (those 525 individuals who responded to 
both the first and second questionnaire), the number of observations is constant. 

Computational formulas used in STSUM are the following: 

1, Number (N) ~ number of observational units for a variable 

N 

2. Total = sum of observational unit values for a variable = SUM X 

A complete description of STSUM is available from The Computation Center The Penn- 
sylvania State Univarsity, University Park^ Penn^lvanta, 
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